Projector Phones: A New Class of Interfaces for Augmented Reality

With the miniaturization of projection technology, the integration of tiny projection units into mobile devices is no longer fiction; therefore, such integrated projectors in mobile devices could make mobile projection ubiquitous. These phones will have the ability to project large-scale information onto any surfaces in the real world, and by doing so, the interaction space of the mobile device can be considerably expanded. In addition, physical objects in the environment can be augmented with additional information, which can support interaction concepts that are not even possible on modern desktop computers today. The authors believe that mobile camera-projector units can form a promising interface type for mobile Augmented Reality (AR) applications, thus, this paper identifies different application classes of such interfaces. In addition, different spatial setups of camera and projector units will have an effect on the possible applications and the interaction space with the focus on the augmentation of real word objects in the environment. This paper presents two examples of applications for mobile camera-projector units and different hardware prototypes that allow augmentation of real world objects. Digital projectors are shrunken to the size of a mobile phone. The next step is to integrate them directly into the mobile device. Up to now several prototypes have been presented, and the first series-production device is already up for pre-order. First prototype phones with integrated projectors already exist and are available on the consumers market but both models are currently not available on the mass market. Such phones could overcome the shortcomings DOI: 10.4018/jmhci.2010070101 2 International Journal of Mobile Human Computer Interaction, 2(3), 1-14, July-September 2010 Copyright © 2010, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. of the small screen and make it possible to present large and complex information like maps or web pages without the need for zooming or panning (Hang et al., 2008). Considering the anticipated widespread availability of phones with integrated cameras and projectors in just a few months, surprisingly little research has been conducted so far to investigate the potential of such a mobile unit (in the following we use the term mobile camera-projector unit as a synonym for a mobile phone equipped with a camera and a projector). We identify different application types based on the spatial configuration of the camera and the projector. As part of this classification, we derive three different application types using of different spatial layouts of cameras and projectors: congruent setups, partially intersecting setups and disjunct setups. Such a classification is useful to structure the design space of mobile camera-projector systems, because we think other researchers can categorize their applications to focus on specific problems and topics of each type. Our approach needs to be elaborated further and more deeply by others researches, but we think it still gives a good framing of this important problem for the usability of mobile camera-projector units, because mobile projector phones and mobile projection is still a young research field. The remainder of this paper is structured as follows. Related work in the field of mobile projection and mobile camera projector units as well as the general interaction with mobile devices is presented in Section 2. Section 3 describes the different application classes of these interfaces. In this conceptual section we also discuss how the spatial layout of the camera relative to the mobile projection unit can affect the characteristics of applications for this new sort of hardware. Next, two example applications are presented: Map Torchlight and LittleProjectedPlanet. Map Torchlight is an application that combines high resolution paper maps with lightweight mobile projection to augment the paper map directly with additional personal and dynamic information. The mobile camera-projector unit is tracked over a paper map and precisely highlights points of interest, streets, and areas to give directions and other guidance for interacting with the map (Schöning et al., 2009). With the LittleProjectedPlanet prototype (Löchtefeld et al., 2009) we explore the possibilities of camera projector phones with a mobile adaptation of the Playstation 3TM(PS3) game LittleBigPlanetTM. The camera projector unit is used to augment the hand drawings of a user with an overlay displaying physical interaction of virtual objects with the real world. Players can sketch a 2D world on a sheet of paper or use an existing physical configuration of objects and let the physics engine simulate physical procedures in this world to achieve game goals. In addition, we discuss the technical setups we used in both prototypes. Finally, we provide some concluding remarks and outline different scenarios where mobile camera projector units can be useful in the future.